Papers with learning rate scheduling method
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning (2023.findings-acl)
Copied to clipboard
Genta Winata, Lingjue Xie, Karthik Radhakrishnan, Shijie Wu, Xisen Jin, Pengxiang Cheng, Mayank Kulkarni, Daniel Preotiuc-Pietro
| Challenge: | Existing methods to handle catastrophic forgetting fail to retain knowledge learnt in the past when sudden shifts occur in training data distributions. |
| Approach: | They propose a learning rate scheduling method that preserves new information without strongly overwriting past knowledge. |
| Outcome: | The proposed method preserves new information without overwriting past knowledge in a multilingual continuous learning framework. |